Pressure optimization for hydraulic-electric hybrid biped robot power unit based on genetic algorithm

Biped robots have attracted increasing attention because of their flexible movement and strong adaptability to the surroundings. However, the small output torque and the weak impact resistance of the motor drive, as well as the large energy consumption of the hydraulic drive limit the performance of the biped robot drive system. Aiming at these shortcomings, an electric-hydraulic hybrid drive system of biped robot was proposed in this paper. The robot platform was designed based on the prototype of the Zhejiang Lab biped robot. The model of the hydraulic drive system and mechanical structure was established to analyze the dynamic characteristic and the load force during walking. The value function reflecting the energy consumption of the hydraulic drive system was proposed. The pressure of the accumulator in the hydraulic power unit was selected as the control parameter. In order to get the minimum value of the value function, so as to reduce the energy consumption of the hydraulic driving system, the control parameters were optimized by using the genetic algorithm. From the simulation results, the proposed optimization algorithm can improve efficiency by 3.49%.

The open valve control system, especially the servo valve controlled hydraulic actuator, has become the main driving mode of the robot hydraulic drive system. The servo valve has a high response characteristic, which meets the requirements of a robot drive system. Also, the control algorithm is mature with a good control effect. For example, the biped robots like BBH from Bath University 11 , NWPUBR-1 from Northwestern Polytechnical University 12 , and the quadruped robots like Hydraulically-powered Quadruped (HyQ) robots from the Institute of Technology in Italy 13,14,15 , SCalf from Shandong University 16,17 and quadruped robot from Yanshan University 18,19,20 are all driven by servo valve controlled cylinders. However, in an open valve control system with a centralized power source, the power unit should always maintain high pressure to ensure that the system has a high response characteristic. When the output force of the actuator is small, the power unit still needs to supply oil at the highest pressure, which leads to a large throttling loss of the valve port. Then the energy efficiency of the system is reduced and the heat generated by the valve is increased.
In order to reduce the energy consumption of the hydraulic drive system, many studies have been carried out in resent years. Hydraulic pump control is an effective way to reduce energy losses, such as model predictive control (MPC) 21 . The control method can reduce energy consumption with a reliable pressure-supply ability. Other effective methods mainly focus on trajectory optimization 22,23,24 and mechanism optimization 25 . However, the limitation includes significant leakage and throttle loss. In order to further increase energy efficiency, the difference between the demand pressure of the actuator and the supplied pressure of the power unit should be reduced to minimize the pressure drop of the control valves. A multistage supply pressure system is one of the approaches. For example, multi-pump hydraulic power unit with a valve matrix is one of the attempts 26 . This power unit can significantly increase the energy utilization rate. But it increases the weight of the hydraulic system. So, the improvement of the total energy efficiency of the robot is limited.
According to the analysis above, based on the biped robot platform proposed by Zhejiang Lab, an electrichydraulic hybrid drive system is proposed in this paper. The drive system makes full use of the advantages of the high efficiency of motor drive and the large power density of hydraulic drive. Aiming at the shortage of large energy loss and low efficiency of the hydraulic system, two accumulators are introduced in the hydraulic power unit to supply multistage pressure. Also, the pressure of the power unit is optimized and the control strategy is put forward to further reduce energy consumption. The proposed approach avoids the use of multiple motorpump units, and can reduce throttle loss of control valves.

System configuration
Mechanical construction. The prototype of the Zhejiang Lab biped robot consists of a trunk, hips, thighs, calves and feet, which are hinged between adjacent body structures. There is a degree of freedom (DOF) of turnover to control the abduction/adduction of the leg and a freedom of yaw to control the direction of the leg between the hip and the thigh. The two DOFs provide no vertical bearing capacity, which reduces the demand torque. So, they are designed to be driven by electric motors.
The hip joint, knee joint and ankle joint each have a DOF of rotation. The driving torque of the hip joint and knee joint is large. The peak value of the driving torque of the two joints exceeds the maximum output torque of the electric motor under the impact load. So, the hydraulic drive is utilized. In the underdrive gait algorithm based on the virtual constraint, the ankle joint only controls the foot to be parallel to the ground, and the required torque is small. So, it is also driven by an electric motor.
In order to reduce the moving parts moment of inertia, actuators should be centrally installed on non-moving parts such as the hip, or on the structures with small rotation radius such as the upper thigh. For this purpose, the turnover and yaw motors are fixedly attached to the trunk and hips respectively. They are mounted coaxially with the rotating shaft. The ankle is driven by an electric motor mounted on the hip through two sets of connecting links. The hip hydraulic cylinder is hinged to the hip and thigh. The knee hydraulic cylinder is hinged to the thigh and calf. The structure of the designed hybrid drive robot is shown inFig. 1. www.nature.com/scientificreports/ Hydraulic system. In order to improve the response of the system and realize accurate position control, the servo valve controlled hydraulic cylinders are utilized as the actuators. The rated pressure of the hydraulic system is 21 MPa. Based on the calculation, the pressure of the hydraulic cylinder is generally no more than 10 MPa under normal walking conditions, and the pressure is high only under the impact load or the instantaneous rapid movement. If the outlet pressure of the pump is maintained at the rated pressure all the time, the pressure difference between the pressure port and the working ports of the servo valve is large, which increases the throttling loss of the servo valve and the input power of the hydraulic pump, resulting in a very low system efficiency. From simulation analysis, if the oil maintains 21 MPa constant pressure, the efficiency of the hydraulic system is only about 10% in the walking condition. In order to improve the efficiency of the system, accumulators with high and low working pressures are used to supply oil to the system. The hydraulic pump charges the accumulator respectively according to the charging state of each accumulator. Each hydraulic cylinder accesses the high-pressure or low-pressure circuit according to the demand pressure, controlled by a directional valve. The principle of the hydraulic system is shown in Fig. 2.
In the system, 11.1 is a low-pressure accumulator and 11.2 is a high-pressure accumulator. The following is an example to introduce the working principle. When the pressure in the hydraulic cylinder 14.1 is lower than the set pressure of the low-pressure accumulator, the directional valve 9.1 connects the cylinder 14.1 to the lowpressure accumulator circuit. At this time, the pressure at the oil inlet P of the servo valve 12.1 is low. When the cylinder 14.1 drives the joint, the oil in the low-pressure accumulator is consumed. On the contrary, when the pressure in the cylinder 14.1 is higher than the set pressure of the low-pressure accumulator, the directional valve 9.1 connects the cylinder 14.1 to the high-pressure accumulator circuit. At this time, the pressure at the oil inlet P of the servo valve 12.1 is high pressure, and the oil in the high-pressure accumulator is consumed when the hydraulic cylinder 14.1 acts. The hydraulic pump 3 charges liquid to the low-pressure accumulator or high-pressure accumulator through the directional valve 6. In this system, the hydraulic pump charges the lowpressure accumulator most of the time. At that moment, the outlet pressure of the pump is equal to that of the low-pressure accumulator. So, the energy consumption of the hydraulic pump can be reduced.
According to the load of the robot, the parameters of the main components are calculated, as shown in Table 1.

Mathematical model
The mathematical models of servo valves, hydraulic cylinders, accumulators and pump are established. For reasons of space limitation, the specific mathematical formula is omitted. Also, the mechanical structure model is established to obtain the load force and energy consumption.  www.nature.com/scientificreports/ Mechanical structure and hydraulic system. The objective of this paper is to analyze the performance of the hydraulic drive system, which drives the hip joint and knee joint. In order to simplify the simulation model, the robot movement is limited to a two-dimensional plane, which has no obvious effect on the analysis results of the hydraulic system. The physical model of the mechanical structure and the hydraulic system is established in AMESim as shown in Fig. 3. In the proposed robot, only the hip joint and knee joint are driven by hydraulic system. The two DOFs of turnover and yaw of the leg, which controls the lateral movement, has little relation with the performance of the hydraulic system. For this reason, the robot movement is limited to a two-dimensional plane, which has no obvious effect on the analysis results of the hydraulic system. The computer used to simulate the model contains an Intel Core i7-9700 K CPU. The geometric dimensioning of the mechanical structure is the same as Fig. 1, while the parameters of the main components are the same as Table 1. Figure 4 shows the topology model of the robot in the simulation. In this paper, the gait algorithm based on virtual constraint is used to control the robot walking within 20 s 27 . The gait algorithm is formulated as a large-scale programming problem. The dynamics constrains are established for the proposed robot configuration. In order to reduce the dimensionality, the system state of the hybrid dynamics model is reconstructed. Following the problem formulation and optimization, the optimized gait is applied to the biped locomotion under the design of feedback control. During the movement, the position of the robot's gravity center is shown in Fig. 5. Figure 5 also shows the projection of the gravity center on the x-t plane and the y-t plane. The sensors in the model include angle, force and pressure sensors. The angle sensors are mounted coaxially with joints to measure the joint angles. Force sensors locate at the end of cylinder rods to measure the output force of the cylinder. Pressure sensors are installed in the cylinders and the outlet of accumulators to acquire the pressure in the hydraulic system. Based on the model, parameters such as joint angle, actuator output force and pressure can be obtained.
Energy consumption. According to the load force, the output power of the actuator can be calculated:   www.nature.com/scientificreports/ where P c_out is the output power of the cylinder, F c is the output force of the cylinder, x p is the displacement of the piston, i= 1, 2, 3, 4 representing left hip, right hip, left knee and right knee respectively. Then, the active work done by the actuator is: where W c_out is the active work of the cylinder, t f is the final moment of the movement. For the specific movement calculated in this paper, t f = 20s.
The input power of a cylinder can be expressed as: where P c_in is the input power of the cylinder, p vP is the pressure of servo valve port P, q vP is the flow rate of servo valve port P. The input power of the hydraulic system can be expressed as: where P in is the hydraulic system input power, T p is the pump input torque; ω p is the pump rotate speed. The energy consumed by the hydraulic system is: where W in is the hydraulic system energy consumption, E a_i is the energy stored in the accumulator at the beginning of the movement; E a_f is the energy stored in the accumulator at the end of the movement; η a is the average efficiency when the hydraulic system charges the accumulator. We take η a = 95%.

Power unit pressure optimization algorithm
The purpose of the proposed algorithm is to reduce the energy consumption of the hydraulic system by optimize the pressure of the power unit. The algorithm also controls the hydraulic pump to charge the accumulators to maintain the pressure within a certain range. A value function is selected to represent the energy consumption as well as the penalty item. In order to take full use of the low-pressure accumulator, the switching time interval of the reversing valve is introduced as the penalty function item. Then the value function is: where t s is the switching time interval; k p is the coefficient of the penalty function item: where t set is the ideal switching time interval of the reversing valve; k ep is the coefficient.
The pressure of the high-pressure accumulator is determined by the rated pressure of the system and the demand pressure of the actuator. There is no need to optimize the pressure of the high-pressure accumulator. Therefore, the pressure range of the high-pressure accumulator is set as 19-21 MPa. That is, the minimum working pressure of the high-pressure accumulator is p ah_l = 19MPa and the maximum working pressure is p ah_h = 21MPa.
(1) www.nature.com/scientificreports/ We assume that the lowest working pressure and the highest working pressure of the low-pressure accumulator are p al_l and p al_h , respectively. In order to reduce the maximum input power of the hydraulic system, the hydraulic pump should charge the accumulators with the approximately constant flow to avoid the instantaneous large flow of oil output. Also, in order to reduce the computational complexity, external disturbances are not considered. The accumulators utilized in the system can supply the robot walk several steps without additional charging. Even though the real consumed flow by the robot changes periodically during walking, the average flow rate during the charging cycle of the accumulators can be considered approximately a constant value. Based on this situation, it is assumed that the actuators consume the oil in the high/low pressure accumulator at a constant rate of q ah_out and q al_out , respectively. The hydraulic pump charges the high/low pressure accumulator at the flow rate of q ah_in and q al_in . It can be obtained by equaling the consuming time to the charging time: From the equations above, we can get q ah_in and q al_in . Then the output flow rate of the pump is: where k s is the input signal of reversing valve. When the pump charges the high-pressure accumulator, k s = 1 . Otherwise, k s = 0.
The power unit control rule is: (1) In the initial state, the pressure of the high-pressure accumulator is the highest set working pressure p ah_h , and the pressure in the low-pressure accumulator is the lowest set working pressure p al_l . (2) When the pressure in the low-pressure accumulator drops to the lowest set pressure p al_l , the hydraulic pump charges the low-pressure accumulator with flow rate q al_in until the pressure reaches and maintains the highest set pressure p al_h . (3) When the pressure in the high-pressure accumulator drops to the lowest set pressure p ah_l , the hydraulic pump charges the high-pressure accumulator. (4) When the pressure in the high-pressure accumulator reaches the highest set pressure p ah_h , the hydraulic pump is switched back to charge the low-pressure accumulator with flow rate q al_in . The rest can be done in the same manner.
The switching rule of the actuator access to the high/low pressure circuit is: (1) When the pressure of the two chambers of the hydraulic cylinder is lower than the minimum set pressure of the low-pressure accumulator p al_l , the hydraulic cylinder is connected to the low-pressure circuit. (2) When the pressure of either chamber of the hydraulic cylinder is higher than the minimum set pressure of the low-pressure accumulator p al_l , the hydraulic cylinder is connected to the high-pressure circuit.
The pressure range of the low-pressure accumulator is determined by the genetic algorithm. The parameters of genetic algorithm are shown in Table 2.
The genetic algorithm is shown below:  With the large-scale computation through the genetic algorithm, the parameters that make the value function obtain the global optimal value can be calculated. Then, the working pressure of the low-pressure accumulator is determined. The penalty function item in the value function ensures that the switching time interval of the reversing valve is not too short. The energy stored in the low-pressure accumulator can be fully utilized. With the determined working pressure, the hydraulic system can achieve a relatively high efficiency.
According to the pressure optimization algorithm proposed above, the data collected during robot walking include the pressure of low-pressure accumulator and high-pressure accumulator, the pressure of each chamber of the hydraulic cylinders, the rotate speed of the pump and the motion of each hydraulic cylinder. The pressure of the accumulators can be measured by the pressure sensors located at the outlet of the accumulators. The hydraulic cylinders are the ones integrated with pressure sensors which measure the pressure in the chambers. The flow that the pump supplied and the cylinders consumed are calculated by the rotate speed of the pump and the joints.
In order to compare the energy consumption, the robot walks along the gait described in Section "Mechanical Structure and Hydraulic System" in different conditions. Also, the external disturbances are ignored to reduce the complexity.

Numerical experiment
In order to verify the effectiveness of the proposed power unit pressure optimization algorithm, the working pressures of the low-pressure accumulator calculated by the optimization algorithm are used. The energy consumption during walking is obtained. The model and parameters, as well as data acquisition process, are the same as mentioned above.
Optimized working pressure. Firstly, the optimized working pressure of the low-pressure accumulator is obtained according to the optimization algorithm. The mean fitness and best fitness during the training process are shown in Fig. 6.
The best solution calculated by the genetic algorithm is [ p al_l ,p al_h ] = [6.2,8.7]. Based on the calculation result, the working pressures of the accumulators are shown in Table 3.
The simulation model described in Section "Mathematical Model" is used to simulate and analyze the robot movement. The input power and output power of each joint are shown in Fig. 7.
The input power of the hydraulic system after optimization is shown in Fig. 8. After optimization, the active work and energy consumption of the hydraulic system is shown in Table 4.
Pressure without optimization. In order to calculate the power and energy consumed by the hydraulic system and provide the comparison for the power unit pressure optimization algorithm, the power unit pressure without optimization control is obtained in this section.
Since the demand pressure of the hydraulic actuator is less than 10 MPa in most of the time, the pressure range of the low-pressure accumulator is set as 10-12 MPa and the high-pressure accumulator is set as 19-21 MPa. In this working condition, the input power and output power of each joint are shown in Fig. 9, the input power of the hydraulic system is shown in Fig. 10 www.nature.com/scientificreports/ Based on the calculation above, the active work and energy consumption of the hydraulic system can be obtained, as shown in Table 5.
Discussion. According to the simulation results, the overall efficiency of the hydraulic system optimized by the optimization algorithm is improved by 3.49%. The efficiency of the knee joint hydraulic cylinder increases by 2.00%-2.60, while the efficiency of the hip joint hydraulic cylinder decreases by 2.33%-3.32. Since the knee joint consumes more energy, the overall efficiency of the hydraulic system is improved.     www.nature.com/scientificreports/ During the switching process of high-pressure and low-pressure power source, it is easy to generate the impact force due to the sudden change of pressure. The impact force increases with the increase of pressure difference between high-pressure and low-pressure power source. The impact force will reduce the stability during movement. Also, the instantaneous flow into the actuator will increase at the impact moment, which will increase the instantaneous power and energy consumption of the joint actuator. According to the comparison between the optimized and not optimized system, the variations of the parameters are shown in Table 6. In Table 6, " + " means that the parameters of the optimized system are larger.
In order to verify the applicability of the algorithm, simulations on different gaits are carried out. The gaits make the robot walk in constant speed during the simulation. The system configuration and parameters are the same as mentioned in Section "System Configuration" and "Mathematical Model". In the original system, the pressure range of the low-pressure accumulator is set as 10-12 MPa and the high-pressure accumulator is set as 19-21 MPa. The proposed algorithm is utilized to optimize the pressure range of the low-pressure accumulator. The simulation results show that the proposed algorithm can improve the overall energy efficiency of the hydraulic system from 2.88% to 3.93%. The detailed simulation process is omitted due to the limitation of the spaces. The simulation results are shown in Table 7.
According to the analysis of the above results, the power unit pressure optimization algorithm has a better energy saving effect on the knee actuator. The reason is that the output force of the knee joint actuator is smaller than that of the hip joint without considering the impact load due to the different joint torque. Therefore, appropriately reducing the low-pressure oil supply pressure of the power unit is conducive for the knee joint actuator to the full use of the low-pressure circuit. This will contribute to reducing the throttling loss and improving the efficiency of the knee joint actuator.
On the other hand, high speed on/off valve is adopted to improve the pressure switching speed of the actuator, which will contribute to improving the response speed and reducing the pressure loss during switching. However, the simulation results show that the switching process will produce large pressure fluctuations. The pressure fluctuations adversely affect stability and increase energy consumption. In order to analyze the influence of the highpressure and low-pressure switching process on the system energy consumption, the simulation model is further modified in this paper. The two-position two-way proportional valve is used to realize the pressure switching. Integrating element and saturation element are added into the switching signal to make the pressure of the servo valve port P rise or fall along a slop signal. The switching time is set as 0.1 s. In this model, the same movement gait is simulated and analyzed. The simulation results show that this method can reduce the impact force caused by pressure switching to a certain extent. But because of the delay of switching, the hydraulic system consumes more high-pressure power source to supply oil, which leads to the significant increase of energy consumption. Therefore, the selection of hydraulic system parameters needs to balance and optimize the contradiction between switching stability and energy consumption. This will be studied in the future work.   www.nature.com/scientificreports/ Additionally, the fluid flow required by the hydraulic actuator during the walking process of the biped robot varies widely. With such flow characteristics, accumulator pressure also presents strong fluctuation. The response characteristics of hydraulic pump and electric motor make it hard for the pump output flowrate to accurately equal to the consumed flow rate all the time. The pump can only provide an average flow rate. When the robot moves steadily with constant walking speed, the amount of oil consumed in each step is basically the same. In this case, the optimization algorithm can maintain the pressure variation range of the accumulator within the set value. The hydraulic system can make full use of the energy stored in the accumulator to achieve a better energy saving effect. However, when the robot's walking speed changes, the average flow consumed by the actuator changes. The accumulator is prone to be under-charged, over-charged, or unable to reach the minimum set pressure, which makes the energy stored in the accumulator can not be fully used and the system energy consumption increases. So, the proposed optimization algorithm is more suitable for constant speed walking.
Finally, in order to verify the robustness of the algorithm, multiple calculations are carried out with the same gait. Due to the randomness of the Genetic Algorithm, the calculation results have fluctuation. The fluctuation range is no more than 0.1 MPa, which has no obvious influence on the energy efficiency of the hydraulic system.

Conclusion
In this paper, an electric-hydraulic hybrid drive system is designed for the biped robot platform of Zhejiang Lab. The optimization algorithm of hydraulic power source pressure based on the genetic algorithm is proposed. Firstly, the model of the hydraulic system and mechanical structure is established. From the simulation analysis of the mechanical structure model, the motion and load characteristics of each joint under a specific action are obtained. Then, a value function reflecting the energy consumption of the hydraulic system is proposed. The value function calculates the energy consumption of the hydraulic system when a specific action is completed according to the pressure setting range of the low-pressure accumulator. A penalty function is introduced to control the switching time of the actuator between high and low pressure. Finally, genetic algorithm is used to calculate the low-pressure accumulator pressure setting range which makes the value function reach the minimum value.
According to the simulation analysis, the efficiency of the optimized hydraulic system is 3.49% higher than that of the unoptimized one. However, since the pressure difference between the high-pressure and low-pressure power sources is larger, it is easy to generate the impact force due to the sudden change of pressure. In order to further reduce the energy consumption of the hydraulic system, parameter matching of the hip and knee actuators should be carried out by combining the robot's geometric scale, locomotive gait and the force. Also, the selection of hydraulic system parameters needs to balance and optimize the contradiction between switching stability and energy consumption.

Data availability
The data that support the findings of this study are available from the corresponding author, [P.Y.Z.], upon reasonable request.